Dexterous robotic manipulation using deep reinforcement learning and knowledge transfer for complex sparse reward?based tasks

This paper describes a deep reinforcement learning (DRL) approach that won Phase 1 of the Real Robot Challenge (RRC) 2021, and then extends this method to more difficult manipulation task. The RRC consisted using TriFinger robot manipulate cube along specified positional trajectory, but with no requirement for have any specific orientation. We used relatively simple reward function, combination goal-based sparse distance reward, in conjunction Hindsight Experience Replay (HER) guide DRL agent (Deep Deterministic Policy Gradient (DDPG)). Our allowed our agents acquire dexterous robotic strategies simulation. These were applied real outperformed all other competition submissions, including those traditional control techniques, final evaluation stage RRC. Here we extend method, by modifying task require maintain particular orientation, while is moved required trajectory. also orient makes unable learn through blind exploration due increased problem complexity. To circumvent issue, make novel use Knowledge Transfer (KT) technique allows learned original (which was agnostic orientation) be transferred (where orientation matters). KT perform extended simulator, which improved average deviation from 0.134 m 0.02 m, 142{\deg} 76{\deg} during evaluation. concept shows good generalisation properties could actor-critic algorithm.